Opinions Regarding the Academy of Managed Care Pharmacy Dossier Submission Guidelines: Results of a Small Survey of Managed Care Organizations and Pharmaceutical Manufacturers

BACKGROUND: In recent years, there has been more emphasis on determining the total value of a drug product, which includes safety and efficacy information and clinical and economic value relative to other therapies. The Academy of Managed Care Pharmacy (AMCP) Format for Formulary Submissions was intended as a tool to assist health care providers in evaluating and selecting drug products. OBJECTIVES: The purpose of this research was to gain the perspectives of a sample of managed care organizations (MCOs) and pharmaceutical manufacturers regarding the AMCP Format submission and evaluation process, as well as their comments on possible future direction for these guidelines as an important part of the formulary decision-making process. METHODS: A random sample of large (greater than1 million lives) and small (less than 1 million lives) MCOs was generated using telephone numbers from the National Directory of Managed Care Organizations database. Pharmaceutical manufacturer respondents were identified from the Pharmaceutical Research and Manufacturers of America (PhRMA) Foundation's Health Outcomes Committee. Telephone interviews were conducted by 2 researchers between September 2004 and October 2005. Respondents from both pharmaceutical manufacturers and MCOs nationwide were familiar with the AMCP dossier preparation and review process, allowing us to compare perspectives from each group. The interview was designed to assess the following key areas: economic models, organizational burden, confidentiality, overall value, and future expectations. RESULTS: Representatives from 20 MCOs and 7 pharmaceutical manufacturers completed the interview; 21 MCO representatives refused to participate, citing company policy. Nearly all (87.5%) of the MCO personnel contacted reviewed dossiers within their organization. However, MCO respondents indicated that only 40% of all drugs they reviewed included dossiers from the manufacturers. For drug evaluation at the level of the pharmacy and therapeutics committee, we found that drugs were compared with a variety of products, with 11 respondents reporting comparisons with a placebo, and all respondents reporting a comparison with at least 1 other branded product. On average, 53.5% of the dossiers MCOs received included budget impact models, and 39.3% included cost-effectiveness analyses (CEAs) or cost-benefit analyses. Of the dossiers with economic models, less than half (46.2%) were deemed adequate. Nearly two thirds of MCO respondents reported that they modified the provided model with their own population statistics, as many reported that manufacturers do not make models directly applicable to their health plan population. The perspectives of the pharmaceutical manufacturers varied dramatically from the MCO respondents with regard to the inclusion of economic models. Five of the 7 respondents indicated that their companies always included an economic model in the submitted dossiers. One respondent indicated that 85% of company dossiers included models, and another reported that 50% of dossiers included CEA models. Both MCOs and pharmaceutical manufacturers commented that organizational burden was high, with 70% of both groups reporting the use of outside consultants to assist in the dossier process. CONCLUSIONS: Overall, findings for this study suggest that awareness of the AMCP Format is high among MCOs and pharmaceutical manufacturers, but aligning objectives between the 2 organization types is necessary. Conceptually, proving a drug value beyond what the U.S. Food and Drug Administration requires is a reasonable request, something most respondents agreed on. However, less than half of all drugs reviewed had a dossier. In contrast to MCO respondents, pharmaceutical manufacturers appear to have a more positive outlook on the role of the AMCP Format in effectively communicating the value of a new drug product.

OBjeCTIve: The purpose of this research was to gain the perspectives of a sample of managed care organizations (MCOs) and pharmaceutical manufacturers regarding the AMCP Format submission and evaluation process, as well as their comments on possible future direction for these guidelines as an important part of the formulary decision-making process.
MeThODS: A random sample of large (>1 million lives) and small (<1 million lives) MCOs was generated using telephone numbers from the national Directory of Managed Care Organizations' database. Pharmaceutical manufacturer respondents were identified from the Pharmaceutical Research and Manufacturers of America (PhRMA) Foundation's health Outcomes Committee. Telephone interviews were conducted by 2 researchers between September 2004 and October 2005. Respondents from both pharmaceutical manufacturers and MCOs nationwide were familiar with the AMCP dossier preparation and review process, allowing us to compare perspectives from each group. The interview was designed to assess the following key areas: economic models, organizational burden, confidentiality, overall value, and future expectations.
ReSulTS: Representatives from 20 MCOs and 7 pharmaceutical manufacturers completed the interview; 21 MCO representatives refused to participate, citing company policy. nearly all (87.5%) of the MCO personnel contacted reviewed dossiers within their organization. however, MCO respondents indicated that only 40% of all drugs they reviewed included dossiers from the manufacturers. For drug evaluation at the level of the pharmacy and therapeutics committee, we found that drugs were compared with a variety of products, with 11 respondents reporting comparisons with a placebo, and all respondents reporting a comparison with at least 1 other branded product. On average, 53.5% of the dossiers MCOs received included budgetimpact models, and 39.3% included cost-effectiveness analyses (CeAs) or cost-benefit analyses. Of the dossiers with economic models, less than half (46.2%) were deemed adequate. nearly two thirds of MCO respondents reported that they modified the provided model with their own population statistics, as many reported that manufacturers do not make models directly applicable to their health plan population.
The perspectives of the pharmaceutical manufacturers varied dramatically from the MCO respondents with regard to the inclusion of economic models. Five of the 7 respondents indicated that their companies always included an economic model in the submitted dossiers. One respondent indicated that 85% of company dossiers included models, and another reported that 50% of dossiers included CeA models. Both MCOs and pharmaceutical manufacturers commented that organizational burden was high, with 70% of both groups reporting the use of outside consultants to assist in the dossier process.
COnCluSIOnS: Overall, findings for this study suggest that awareness of the AMCP Format is high among MCOs and pharmaceutical manufacturers, T he formulary system is intended as a tool to assist health care providers in evaluating and selecting drug products. In the past, the evaluation of drugs by managed care organizations (MCOs) for inclusion on the formulary had a primary focus on safety and efficacy issues and a secondary focus on the overall cost-effectiveness of the product. 1 There has been a move in recent years to consider all available information in an evidence-based formulary system. In 1994, Regence BlueShield created a set of guidelines for reviewing and but aligning objectives between the 2 organization types is necessary. Conceptually, proving a drug value beyond what the u.S. Food and Drug Administration requires is a reasonable request, something most respondents agreed on. however, less than half of all drugs reviewed had a dossier. In contrast to MCO respondents, pharmaceutical manufacturers appear to have a more positive outlook on the role of the AMCP Format in effectively communicating the value of a new drug product. Further steps need to be taken to improve acceptance and integration of the AMCP Format.

What is already known about this subject
• There is little known about how MCOs are receiving and incorporating the drug product dossiers. The experience of 1 MCO showed that it received dossiers in 58% of the requests, and pharmacoeconomic models were included in 68% of the dossiers received. However, most dossiers had limited utility because they were not unlocked interactive economic or budget-impact models that permitted use of MCO-specific data. 14

What this study adds
• This research provides perspectives on the AMCP Format from both MCO and pharmaceutical manufacturer representatives.
• Less than half of MCO respondents found the pharmacoeconomic (PE) models to be adequate. Two thirds of MCO respondents reported that they used the PE model provided with the dossier to examine the results using data from their own MCOs.
Opinions Regarding the Academy of Managed Care Pharmacy Dossier Submission Guidelines: Results of a Small Survey of Managed Care Organizations and Pharmaceutical Manufacturers evaluating drug products, 2 which has since been espoused by the Academy of Managed Care Pharmacy (AMCP) and is currently considered the gold standard for evidence used by MCOs in considering drugs for the formulary. 3 The main intentions of these guidelines are as follows: (1) decisions regarding a medication' s inclusion on a formulary will be based on the overall value that medication brings to a specific population, and (2) the value argument will be based on good scientific evidence, including pharmacoeconomic models as an integral part of formulary review. The AMCP Format for Formulary Submissions guidelines encourage pharmaceutical manufacturers to submit a structured dossier that includes clinical and economic data from both published and unpublished studies and a disease-based economic model that predicts the product' s impact on health outcomes and economic consequences for the specific health plan. Because the guidelines are promoted as a template rather than a mandate, individual MCOs may implement them with modifications to fit their organizations, resulting in a wide variety of outcomes. The Format is now viewed as an industry standard, but little is known about how MCOs and drug companies are receiving and incorporating the guidelines. [4][5][6] Of primary concern to many has been the length and preparation involved in the dossier format. Regence BlueShield reported that the time required for its staff members to prepare review summaries for the pharmacy and therapeutics (P&T) committee decreased over time as they became more experienced with the Format. 7 To date, few studies have specifically investigated the use of pharmacoeconomic models in formulary decision making. Watkins et al. 8 used a case-study approach (CORE diabetes model) to show how the use of an economic model can affect formulary decisions. The model demonstrating the effects of exenatide versus comparators on long-term disease burden and costs was included as part of the formulary monograph. The pharmacy staff members of the health plan manipulated the model to project cost outcomes, and in this case, the decision to include the medication on the formulary partially owed to these results.
Earlier, Sullivan et al. reported the results of a roundtable discussion in 2004 involving 12 MCO representatives and pharmacy benefit managers (PBMs). 9 The main findings from their discussion, which focused on P&T committee reactions to pharmacoeconomic models, included an emphasis on clinical meaningfulness over statistical significance and on increasing the transparency of models so that P&T committees could easily adapt the model for their plan population.
Olson et al. conducted a survey of 20 pharmacoeconomic researchers from U.S. pharmaceutical and biotechnology companies to better understand industry views of how MCO decision makers perceive economic models. 10 Findings suggest that models do affect health policy decisions. Olson et al. also reported important findings regarding intellectual property and confidentiality. Eighty percent of their survey participants would not leave electronic copies of their models with MCOs for a variety of reasons. The 20% who did provide electronic copies required signed confidentiality agreements, reinforcing the importance of the role of confidentiality guidelines in dossier submissions.
Previous research has shown the growing importance of the role of pharmacoeconomics in drug research and development 11 as well as in formulary decisions. 7,8 Little research, however, has investigated the impact of the AMCP dossier in these areas or assessed both pharmaceutical firms and MCO views simultaneously. In the current study, we conducted phone interviews with 35 people from both pharmaceutical manufacturers and MCOs nationwide who are familiar with the AMCP dossier preparation and review. These interviews enabled us to compare perspectives from the respondents of each group. The interviews were designed to assess the following key areas: economic models, organizational burden, confidentiality, overall value, and future expectations. These research topics expanded on those used in previous studies of this nature. [10][11][12] This article reports on both the MCOs' and pharmaceutical manufacturers' responses for each key area and provides possible explanations for the discrepancies that were noted. We conclude by commenting on the policy implications of these findings and recommending future action to improve the nature of the AMCP guidelines in the formulary decision-making process.
nn Methods Interview A structured interview protocol and questions tailored for MCOs and pharmaceutical companies were developed and pilot tested for use in this project (see Tables 1 and 2). The interview protocol was designed to facilitate information exchange quickly to reduce respondent burden. Each interview lasted approximately 15 minutes. Respondents were asked to recall the past 6 months and comment on the review and formulary decision-making processes. Additionally, respondents were given the opportunity to provide open-ended commentary on the AMCP Format after answering the structured survey questions. Interview content and protocol were reviewed and approved by the appropriate human subjects review committee. Two investigators conducted the interviews using this protocol.

Study Population
Participants were contacted by phone between September 2004 and October 2005.
MCOs. A random sample of large (>1 million lives) and small (<1 million lives) MCOs was generated using phone numbers from the National Directory of Managed Care Organizations' (NDMCO' s) database. NDMCO is a publicly available database that provides contact information and company profiles for MCOs. We attempted to sample from both large and small q None q 1-3 q 4-6 q 7-9 q 10 or more 5a) Please estimate the number of those reviewed in the last 6 months that included dossiers from manufacturers? _____ (If respondent has difficulty coming up with number, probe with the following formatted choices:) q None q 1-3 q 4-6 q 7-9 q 10 or more 5b) How many of these reviews have included an economic model that you considered adequate? q 100% q 75% q 50% q 25% q None 5c) How many of these included a budget impact model? q 100% q 75% q 50% q 25% q None 5d) How many of these included an incremental cost-effectiveness or costbenefit ratio? q 100% q 75% q 50% q 25% q None 6) How many times in the last 6 months has your staff developed an original economic model specific to a formulary decision? _________ (If respondent has difficulty coming up with number, probe with the following formatted choices:) q None q 1-3 q 4-6 q 7-9 q 10 or more If None, GO to 6A. If one or more, SKIP to 7.
6a) If not in the last 6 months, when is the last time you developed your own model for formulary decision making? q 6 months-one year ago q More than one year ago q Never 7) When thinking about the most recent dossier, did you modify the provided model for use with your own population? q Yes If yes, what was the class of the drug that the last population modification was built for? ________________________ q No 8) When thinking about the most recent dossier, did you vary the assumptions of the model outside the bounds of the model that you have been given? q Yes GO to 8a q No SKIP to 9 8a) When thinking about the most recent dossier, which assumption made the most important difference? q Pricing q Market share q Probabilities of events q Quality-adjusted life-years q None of the above companies, since organization size may affect the ways in which health plans evaluate and respond to dossier submissions. One hundred ninety-one pharmacy directors (or equivalent position) were contacted to complete the MCO telephone interview. A substantial number of telephone numbers were missing or inaccurate, 21 organizations refused to participate in the survey per company policy, and 64 pharmacy directors were unreachable. Nine (7 small, 2 large) MCO operators referred us directly to a PBM when we inquired about speaking to anyone involved in formulary decisions. There were no differences in company size between responders and nonresponders. Pharmaceutical manufacturers. Members of the Pharmaceutical Research and Manufacturers of America (PhRMA) Foundation' s Health Outcomes Committee were contacted to participate in the survey of pharmaceutical manufacturers. This committee is made up primarily of directors of health outcomes departments of pharmaceutical companies and consists of approximately 15 members, all of whom have experience with multiple dossiers developed in accordance with dossier guidelines. These respondents are likely representative of a very knowledgeable and involved group of pharmaceutical manufacturers.
nn Results Respondent Information MCOs. A total of 28 (18 small, 10 large) MCO representatives agreed to participate in the telephone interview. Four (3 small, 1 large) of the 28 respondents reported that their organizations contracted formulary decisions directly to a PBM, and these interviews were promptly ended. Three respondents (11%) reported that their organizations did not use dossiers, and 1 individual was not personally involved in the review process, thus reducing our final sample to 20 (14 small, 6 large). Therefore, our final sample was approximately 10% of the original 191 MCO representatives that we attempted to contact.
Fifteen of those interviewed (75%) held the position of pharmacy director, while the remaining 5 (25%) had similar titles (chief pharmacy officer, pharmacy manager, or vice president of pharmacy). The most common responsibilities of the MCO respondent included evaluating the submitted dossier, compiling data from the health plan for P&T committee use, and presenting staff recommendations at P&T meetings.
Pharmaceutical manufacturers. Seven pharmaceutical manufacturer interviews were completed. Four respondents were directors of health economics and outcomes research,

9)
To what do you usually compare the drug under review? Placebo? q Yes q No At least one branded product?
q Yes q No If yes, how do you select the branded product? q Branded product with greatest market share within class q Cheapest branded product in class q Branded product going generic soonest q Other _______________________________________ 10) Within your organization, who develops and evaluates economic models? (CHECK ALL THAT APPLY) q P&T committee q Pharmacy Director q Pharmacist q Statistician q Pharmacoeconomist 10a) Approximately how many full-time employees (FTEs) have been involved in the development and review of dossiers within the last 6 months? q <1 q 1 q 2 q 3 q 4 q 5 or more 10b) Have you increased staff in the last 2 years to assist in dossier review? q Yes q No 10c) Have you used outside experts or consultants to help you in your review? q Yes q No 11) Does the P&T committee review the economic model? q Yes q No 12) How much time do you give the manufacturer to respond to a request for a dossier? q More than 4 months prior to product review q 2-4 months prior to product review q 1-2 months prior to product review q Less than 1 month 13) How often do the manufacturers meet the timeline? q 100% of the time q 75% q 50% q 25% q Never 14) What assurances are provided to manufacturers regarding the confidentiality of requested dossiers? (CHECK ALL THAT APPLY) q Only staff involved in reviewing are allowed access to the dossier q Only staff reviewing and P&T committee are allowed access to the dossier q The dossier will not be made public q No assurances are provided MCO = managed care organization; P&T = pharmacy & therapeutics. q None q 1-3 q 4-6 q 7-9 q 10 or more 5a) How many of these dossier submissions included an economic model? q 100% q 75% q 50% q 25% q None 5b) How many of these dossier submissions included a budget impact model? q 100% q 75% q 50% q 25% q None 5c) How many of these dossier submissions included an incremental costeffectiveness or cost-benefit ratio?  Table 3 shows that, across all respondents, the drug classes of dossiers most commonly submitted and reviewed in the 6 months before the interview were (1) cardiovascular, (2) lifestyle (including erectile dysfunction, incontinence, dermatology, irritable bowel syndrome), (3) pain, and (4) antibiotics.
On average, 26.5 individual drugs were reviewed over the 6 months before the interview (small MCO = 23 drugs; large MCO = 36). There appeared to be 2 common approaches to formulary reviews at the P&T level. Individual drug candidates were either specifically evaluated at the time of request (most likely at product launch), or the drug was evaluated as part of a general therapeutic class review scheduled to incorporate the new U.S. Food and Drug Administration (FDA)-approved agent. This variation in P&T committee meetings to review drugs may explain the large variation in the number of drugs reviewed, with 7 (35%) respondents reporting less than a dozen products reviewed over the past 6 months, and 4 (20%) reporting more than 50 products. Our results were similar to those found by Sullivan, whose discussions also revealed variability in the frequency of P&T committee meetings. 9 Pharmaceutical manufacturers. Pharmaceutical industry respondents reported that they submitted dossiers to MCOs for approximately 3.9 (range 2-6) individual drugs in the 6 months before the survey interview.

Drug Comparisons
MCOs. For drug evaluation at the P&T level, we found that drugs were compared with a variety of products, with 11 respondents reporting comparisons with a placebo, and all respondents reporting a comparison with at least 1 other branded product. For new drugs in a new class, comparisons were made with a placebo, since few companies conduct head-to-head studies before product launch. 11 Products in the same pharmacologic class or therapeutic class were always compared with the other branded products, with the majority of comparisons between MCOs. On average, respondents reported that 53.5% of dossiers received included budget-impact models and 39.3% included cost-effectiveness analyses (CEAs) or cost-benefit analyses (CBAs). A recent study by Spooner et al. 14 found that pharmacoeconomic models were included in 68% of the dossiers received. Table 4 shows that, of the dossiers with economic models, respondents from the current study believed that nearly half (46.2%), on average, contained adequate economic models. There was a notable difference between small and large MCO respondents, with small plans reporting that 59.5% of dossiers contained an adequate economic model, and large plans reporting an adequate model only 9% of the time. The difference between large and small MCOs for the percentage of dossiers containing an adequate economic model might be a function of the level of expertise in evaluating such models. If smaller health plans are less familiar with economic models or lack the staff to assist in evaluation, they may be more likely to deem a model adequate, whereas a larger plan with more staff and/or experience may not regard it as such. Alternatively, the needs of smaller MCOs might be more modest in dossier reviews than might those of the larger organizations. The capacity of MCOs to adequately interpret and use cost-effectiveness information is unknown. Smaller organizations might not have the resources available for staff training or might prefer contracting out to consultants, such as academic centers.
Fifteen (75%) MCO respondents reported being involved in some capacity with developing or evaluating the economic drugs currently on the formulary.
Most MCO respondents commented at the end of the interview that their primary focus was on clinical efficacy and safety when they made comparisons. If products were equal on these measures, the decision to adopt the medication was then based on financial considerations. Others indicated that formulary decisions rely heavily on rebate options known commonly as the "bid grid," where adding another drug to a therapeutic class decreases the existing rebate structure. The "bid grid" is a tool to determine how the existing rebate structure for competitors' products will change if the drug being considered is added to the formulary in that particular therapeutic class.
Our findings are similar to results from a focus group study by Delate et al. involving P&T committee members. 13 Participants in the Delate study ranked the importance of factors for making drug coverage decisions using a 5-point scale (1 = not important, 5 = very important). Efficacy was ranked as the most important factor (mean = 4.9), followed by cost-effectiveness (4.4), cost (4.0), and safety (4.0). 13 Pharmaceutical manufacturers. Pharmaceutical industry respondents reinforced the MCO results regarding drug comparisons. Five of the 7 companies responding indicated that their dossiers included comparator information with a placebo. All respondents reported providing comparisons with at least 1 branded product, which typically involved the product with the greatest market share within its class. Two of the 7 respondents (28.5%) reported that their companies provided information on all available comparators.

Drug Classes of Dossiers Submitted/Reviewed in the 6 Months Prior to Interview
Opinions Regarding the Academy of Managed Care Pharmacy Dossier Submission Guidelines: Results of a Small Survey of Managed Care Organizations and Pharmaceutical Manufacturers models, while 6 reported that they had a pharmacoeconomist on staff to assist in dossier reviews. In the 6 months before the interview, approximately 8 (40%) MCOs, on average, reported that they had developed original economic models specific to formulary decisions. It is difficult to assess the degree of sophistication involved in these models from interview responses alone. However, on the basis of additional comments from respondents, it appears that the complexity varied drastically from simple utilization or rebate models to more complex costeffectiveness models. Thirteen of 20 (65%) MCO respondents reported that they modified the provided model with their own population statistics, as many reported that manufacturers do not make models directly applicable to their health plan population (Table 4). Many commented that if the manufacturers tailored models to reflect situations specific to individual MCOs, it would make the models much more beneficial. These findings corroborated those of Sullivan et al., in which respondents urged transparency in economic models so P&T committees could easily input their plan demographics into the model. 9 Similarly, about one third (6 of 20) of respondents indicated that they commonly varied the assumptions of models that permitted such testing. The relatively small proportion who conducted these analyses may believe that small changes in an economic model will not have a major impact. Of those who did vary assumptions, they made the key drivers of the models the price of the products and the probability of events associated with treatment. None suggested that assumptions regarding quality-adjusted life-years (QALYs) were of critical importance to the model outcome.
Many MCO respondents complained during their interviews that models provided by drug companies were unnecessarily awkward and difficult to manipulate. Additionally, most commented during the interview that they would rather see simple and easy-to-follow models relevant to specific decisions. Another reason to provide less complicated models is that P&T committees comprise physicians and pharmacists who have not been trained to evaluate cost-effectiveness models. Most MCOs reported that economic models were commonly presented to the committee in a summary review format, with technical aspects omitted. Similarly, roundtable participants in the Sullivan study commented that economic models should be presented in a format comprehensible by P&T committee members. 9 Specifically, they recommended emphasizing clinically meaningful findings over statistically significant results. Pharmaceutical manufacturers. The perspectives of the pharmaceutical manufacturers interviewed varied dramatically from the MCO respondents with regard to the inclusion of economic models. Five of the respondents indicated that their companies always included an economic model in the submitted dossiers. Of the remaining 2 respondents, 1 indicated that 85% of their dossiers included models, and the other reported that 50% of their dossiers included CEA models.
Only 1 of the 7 pharmaceutical manufacturer respondents reported tailoring the model to the requesting health plan' s population. The majority (70%) of MCOs reported modifying the model in-house with their own population. Most pharmaceutical respondents reported basing models on national prevalence rates, while 1 individual was under the impression that MCOs preferred to not provide plan-specific information. On this subject specifically, there appears to be a clear disconnect between MCOs and pharmaceutical

Summary of Responses on Economic Models
manufacturers. Better communication from the MCOs regarding what is expected or desired will enable manufacturers to provide more specific and, hence, more valuable models. This was demonstrated in the Watkins study, in which effective communication between manufacturer and payer resulted in the decision to include exenatide on the formulary. 8 Only 3 pharmaceutical manufacturers' respondents varied the assumptions of the provided model. Assumptions that made the most difference according to pharmaceutical manufacturers were a combination of pricing and outcome (i.e., probabilities of events or QALYs).

Organizational Burden
The additional burden on staff and financial resources continues to be a common complaint of the AMCP dossier format for both MCOs and pharmaceutical firms. 5 MCOs. In this sample, the majority (16 of 20) reported having a pharmacy director involved in developing and evaluating economic models, while 13 of 20 reported P&T committee involvement. The average number of full-time equivalent (FTE) staff members involved in the dossiers in the past 6 months was 5.2. Only 5 of the MCOs reported hiring additional in-house staff within the past 2 years to assist in the dossier review, although 5 of 20 individuals (25%) mentioned, in open commentary, that more help was desired. Fourteen (70%) of the MCOs interviewed used outside experts to assist in dossier review in the previous 6 months. Fifty percent of the large companies indicated that they used outside consultants, while 71% of the small companies enlisted the aid of outside sources.
Pharmaceutical manufacturers. All 7 pharmaceutical company respondents reported building models both in-house as well as with the aid of outside consultants. An average of 1.43 FTEs were involved in developing and reviewing dossiers within the past 6 months. Only 1 of the pharmaceutical manufacturer respondents in our sample reported hiring additional company employees within the past 2 years to assist in the dossier submission process.

Dossier Timelines
MCOs. MCO respondents reported that the timelines related to requests for dossiers ranged anywhere from no deadline (3 of 20 respondents) to 6 months (1 of 20 respondents) before product review. Four of 20 respondents (20%) were not aware of an imposed deadline because they were not personally involved in dossier requests. On average, the MCO respondents who request dossiers reported that manufacturers met the MCOimposed deadlines approximately 83% of the time.
Pharmaceutical manufacturers. Five of the 7 pharmaceutical company respondents were familiar with the timelines associated with requests. Similar to the MCOs, these pharmaceutical manufacturers reported that the deadlines imposed on dossier requests ranged from no deadline to 6 months before product review. Pharmaceutical manufacturers reported meeting the deadline about 95% of the time, which was slightly higher than the figures (84%) reported by MCOs. One pharmaceutical manufacturer respondent commented that the company proactively develops the principal components of the dossiers before product launch (as opposed to waiting for a request) and is therefore more likely to meet timelines.

Confidentiality
Dossier confidentiality is a great concern for pharmaceutical manufacturers, as they are often fearful their proprietary information will be made public. 5 MCOs. Half (10 of 20) of the MCOs reported giving verbal assurances to the manufacturers regarding confidentiality, while the other half reported that no assurances were made.
Pharmaceutical manufacturers. Only 5 of the 7 respondents (71%) were familiar with the confidentiality issues associated with dossier submission. The perspectives of these 5 manufacturers were fairly consistent with the MCO reports on confidentiality. One of the pharmaceutical manufacturers reported that the company requires MCOs to sign a legal confidentiality agreement, but that was not the norm. Four pharmaceutical company representatives reported that they were given verbal assurances by the MCOs that the dossier would not be made public, and 1 reported that no assurances were made. Four of the 5 firms commented on the need for more clearly documented confidentiality guidelines.

Overall Value to Formulary Decisions
MCOs. MCO respondents held various opinions regarding the overall value of the AMCP dossier format. The majority of these opinions were expressed as open-ended comments at the end of the structured survey questions. Some have made the formal decision to include dossiers in their decision-making process, while others do not use them because they fail to see the value they provide, primarily because of perceived bias on the part of the pharmaceutical manufacturers.
In a recent article by Fullerton and Atherly, 7 Regence BlueShield stated that, between January 1998 and June 2000, drugs with useful economic models were approved 75% of the time versus 54% for all drugs reviewed in that time period. Some of the major criticisms (in order of frequency) involve the applicability, length, complexity, and potential bias in models developed by pharmaceutical manufacturers as a key component of the dossier format. Many MCO respondents commented that the models were created to produce desired results, and their construction should be scrutinized. Other respondents reported that because the manufacturers have clearly biased views, their organizations do not give the economic model portion of the dossier reviews much weight in the formulary decision process. On this basis, one may assume that manufacturers with such information may hesitate to put much effort into models if they Opinions Regarding the Academy of Managed Care Pharmacy Dossier Submission Guidelines: Results of a Small Survey of Managed Care Organizations and Pharmaceutical Manufacturers believe they are likely to be overlooked or disregarded.
Pharmaceutical manufacturers. In general, pharmaceutical manufacturers felt that the AMCP Format provided an opportunity to convey the value of a product, including more latitude for economic analyses and arguments. Nearly all (95%) respondents in Olson' s survey of industry pharmacoeconomists reported that an economic model played a role in improving product positioning on formularies at least once in their experience. 10 The results from the study by Spooner et al. showed no relationship between dossier receipt and placement on the formulary. 14 In fact, products that included a dossier submission were less likely (0%) to be placed on the formulary at the second (preferred) copayment tier compared with those without a dossier (33%).

Expectations for the Future
Version 2.1 of the AMCP Format (which became available in April 2005) included changes encouraging clarity in presenting model results and a need to clearly differentiate between cost-effectiveness and budget-impact models. 15 Three of the pharmaceutical manufacturers interviewed believe the new AMCP guidelines will improve the submission process, while the other 3 expected that the guidelines will not affect the current process. Since the Centers for Medicare and Medicaid Services have publicly stated their intention to develop a framework for formulary management based on existing national standards and guidelines (such as those established by AMCP), 16 we asked participants about their expectations in this area. Five of the 7 respondents (71%) reported having already adapted the dossier for Medicaid, and 3 of these (60%) reported that the dossiers had been helpful in informing discussion and negotiation with Medicaid. All manufacturers interviewed believed that prescription drug plans (Medicare Advantage and PDPs) would begin requesting dossiers for their formulary reviews in the near future.

nn Discussion
It was our intention at the beginning of this project to determine the extent to which the AMCP Format was being used within MCOs and drug companies and what impact the dossiers might have on formulary decisions. The data gathered provide important descriptive information on the similarities and differences in perceptions of the respondents who represented the 2 organization types. In addition to giving specific survey responses to structured questions, many interviewees provided comments regarding their overall perspective of the AMCP dossier process.
Overall, findings for this study suggest that awareness of the AMCP Format is high among persons in MCOs with drug formulary decision-making responsibility and among employees of pharmaceutical manufacturers involved in preparing product dossiers, but it is necessary to align objectives between the 2 organization types. Conceptually, proving the value of a drug beyond what the FDA requires is a reasonable request; this was agreed on by almost all respondents in one form or another. However, less than half of all drugs reviewed had a dossier. In contrast to the MCO respondents, pharmaceutical manufacturer respondents appear to have a more positive outlook on the role of the AMCP Format in effectively communicating the value of a new drug product. Further steps need to be taken to improve acceptance and integration of the AMCP Format into the formulary decision-making process.
The different perceptions of the manufacturers and MCOs seem most acute in the use of the Format's economic model. Although safety and efficacy information is more widely accepted by MCOs, many of them have attitudes of distrust and skepticism with the economic analysis portion. It appears that there are 2 different pharmaceutical company perspectives-1 that provides a modifiable model for MCOs to adjust as they see fit, and 1 that provides all the answers with a very specific, tailored model. The latter may reflect the belief that the MCOs will not adjust model parameters or might not have staff capable of developing plan-specific data and of appropriately altering the model parameters.
Our evaluation supports past research that suggests that although many pharmaceutical companies submit dossiers to P&T committees, very few follow the AMCP Format rigorously; most often omit the economic model. 9,14 While Olson found that MCOs sought model simplicity, ability to customize the model, and transparency, 10 it appears that submissions addressed in our research are still lacking in these elements, at least in the perception of the MCOs surveyed. Just more than half of the pharmaceutical company respondents reported including the entire model in the dossier, while others included only a summary but offered the model in its entirety on request.
The usefulness of the economic model becomes a key point as Medicaid pharmacy managers gain more experience with their use in formulary decision making because their interests may be different from those of large MCOs. Most important, Medicaid agencies may have even more acute staff capacity constraints that affect their ability to analyze a model in detail. Models for Medicaid agencies may need to focus on special populations, although this may be a rather steep challenge for pharmaceutical companies because of limited data for populating key elements of the model.
On the basis of our findings, we make the following recommendations: 1. MCOs and pharmaceutical manufacturers engage in continuous communication. MCOs must take a proactive role in initiating dossier requests from pharmaceutical manufacturers. Additionally, MCOs must be willing to provide feedback regarding the perceived shortcomings of the dossier submissions to permit pharmaceutical manufacturers to better meet MCOs' expectations.
2. MCOs need to devote additional resources to evaluating drug dossiers. MCOs need to employ reviewers who have the capability to discern potential model biases as well as the appropriateness of the model in their member populations. Economic models could provide insight into other business processes and policies, but it' s not clear that MCOs have the necessary staff capacity to maximize these opportunities. 3. Pharmaceutical manufacturers need to adopt a perspective of transparency, consistency, and flexibility in their modeling efforts. The principal purpose of the dossier is to determine the drug' s place in treatment, and manufacturers need to consider the dossier as part of a broader strategy toward appropriate therapeutic decision making in general. 4. Because many MCOs now forbid sharing the information discussed in our interview, AMCP needs to promote an anonymous reporting system for both MCOs and pharmaceutical manufacturers to discuss the usefulness of the Format in their efforts to continue to refine their development and evaluation processes. Such actions would allow the AMCP Format to function as intended-providing increased access to information about new drugs, facilitating the formulary decision-making process, and generating higher levels of confidence in the decisions made.

Limitations
The first and main limitation of this study was the small sample size, which makes it difficult to generalize our findings. Nevertheless, our data provide information on how the guidelines are being used and integrated, which is an area that needs additional research. [4][5][6] Second, because our pharmaceutical manufacturer respondents were chosen from the PhRMA Foundation' s Health Outcomes Committee and were therefore very knowledgeable and involved, these respondents may not reflect pharmaceutical manufacturers as a whole. This fact may also make it more difficult to compare their answers with those of MCO respondents, who were selected at random. Third, we did not assess the perspective of the PBMs, which would have provided additional insight into the use of the AMCP Format in formulary decisions in the United States.
Fourth, it is possible that individuals who participated in the interview were "gaming" their answers or were not entirely forthcoming about their practices. We attempted to minimize this possibility by developing questions that would require the individual to have prior knowledge of terms and concepts to respond appropriately. A fifth limitation of this study design is that it was retrospective in nature and relied on the respondents' recall.
Sixth, selection bias must be considered, since those who volunteered their time to participate in the interview process could have been more likely to praise or criticize the dossier process. We anticipate that the ability to gather future survey information on this topic will become more difficult since many MCOs and PBMs now have policies and procedures that prohibit sharing this type of information. Therefore, our data are valuable, especially if future studies of this nature are not well received.

nn Conclusions
Findings from our current research assess perceptions regarding the AMCP Format and give us insight into possible process improvements.
As evidenced by open-ended comments from survey respondents, the current research reinforces the need for increased staffing and training, improved collaboration with individuals and companies experienced in pharmacoeconomic modeling, and improvements in model consistency, transparency, and flexibility. Additionally, our findings suggest the need for greater communication between model developers and users so they have similar expectations.

DiSCLOSureS
Funding for this study was provided by an unrestricted research grant from Pfizer, Inc., and was obtained by author Michael B. Nichol. Nichol is a consultant to Pfizer Inc., and authors Dennis H. Honda and Roma Tretiak are employed by Pfizer, Inc. The coauthors disclose no potential bias or conflict of interest relating to this article. Nichol served as principal author of the study. Study concept and design were contributed by Nichol, Honda, and Tretiak, with input from author Tara K. Knight. Data collection was the work of Knight and author Joshua Epstein; data interpretation was primarily the work of Nichol, with a substantial contribution from Knight and Epstein. Writing of the manuscript was primarily the work of Nichol, with a substantial contribution from Knight and Epstein; its revision was the work of Nichol, Knight, Honda, and Tretiak.